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Purpose: We investigated the merit of ultrasound (US) features and BRAF V600E mu- 
tation as an additional study of cytology and compared the diagnostic performances 
of cytology alone, cytology with US correlation, cytology with BRAF V600E mutation, 
and a combination of cytology, US, and BRAF V600E mutation all together. Materials 
and Methods: This study included 185 patients (mean age, 48.4 years; range 20-77 
years) with 191 thyroid nodules who underwent US-guided fine-needle aspiration 
(FNA) with an additional BRAF V600E mutation test. Three radiologists highly experi- 
enced in thyroid imaging retrospectively reviewed US images and classified each 
nodule into two categories (positive for malignancy or negative for malignancy). In- 
terobserver variability (IOV) of US assessment between the three readers was esti- 
mated using the generalized kappa statistic of Landis and Koch. We also calculated 
the diagnostic performances of these studies. Results: There were 131 cases of malig- 
nancy (131/191, 68.6%) and 60 cases of benign nodules (60/191, 31.4%). In terms of 
IOV of US assessment, the generalized kappa value was 0.242, indicating fair agree- 
ment was reached. The combination of cytology with BRAF V600E showed higher 
specificity (100%) and positive predictive value (PPV) (100%) compared to the com- 
bination of cytology, BRAF V600E , and US (specificity 28.3%, 66.7%, 68.3%; PPV 
74.6%, 86.6%, 86.8%, respectively; pKO.001). However, cytology with BRAF V600E 
showed lower sensitivity (84.7%) than cytology with BRAF V600E and US (96.2%, 
98.5%, 95.4%, respectively; /K0.001). Conclusion: Considering the diagnostic per- 
formance and low reproducibility of US, the combination of FNA with BRAF V600E is 
the most reliable and objective method for diagnosing thyroid malignancy. 
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INTRODUCTION 



The advance of high-resolution ultrasound (US) has led to the discovery of a greater 
number of thyroid nodules. US-guided fine-needle aspiration (FNA) has been wide- 
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ly used as the main diagnostic method for patients with thy- 
roid nodules. However, this method entails some limitations, 
such as nondiagnostic, false-negative and false-positive re- 
sults, the extent of which depends on the skills and accuracy 
of the operator and interpreting cytologist. 13 To overcome the 
limitations of FNA, several molecular factors have been add- 
ed to improve the diagnostic accuracy of US-guided FNA. 4 - 5 
The BRAF V600E mutation is one of the most commonly used 
molecular factors therein and has a high positive predictive 
value (PPV), especially in patients undergoing indeterminate 
FNA. The combination of cytologic diagnosis and testing for 
the BRAF V600E mutation has improved the overall diagnostic 
performance of US-guided FNA. 6 However, checking for the 
BRAF V600E mutation can cause misinterpretation due to false 
positive results-although this occurs rarely-as well as an ele- 
vation of medical costs. 7 - 8 

The correlation of US features with FNA results helps 
overcome the limitations of FNA 9 ~ n Even when cytologic 
results were the same, the malignancy rate was higher when 
nodules had suspicious US findings. 29 This demonstrates 
how US features and FNA results can complement each 
other. US does not involve additional medical costs, but can 
be a very subjective diagnostic tool and dependent on the 
skills and experience of the performer. 1213 

In this study, we investigated the merit of US features and 
BRAF V600E mutation as an additional study of cytology and 
compared the diagnostic performances of cytology alone, cy- 
tology with US correlation, cytology with BRAF V600E muta- 
tion, and a combination of cytology, US, and the BRAF V6CI0E 
mutation all together. 



MATERIALS AND METHODS 



This retrospective study was approved by the Severance 
Hospital Institutional Review Board, and the need for in- 



formed consent for inclusion in this study was waived. Writ- 
ten informed consent for US-FNA and BRAF V600E mutation 
analysis was obtained from all patients included in this 
study, prior to all procedures. 

Patients 

This study was performed at our institution (a referral center) 
from December 2010 through January 2011. During this pe- 
riod, 373 patients with 385 nodules underwent FNA with an 
additional BRAF V600E mutation test. Of the 385 nodules, we 
excluded 26 nodules in 25 patients that were less than 5 
mm in the longest diameter, because these nodules inherent- 
ly have a high false positive rate on US 3 ' 4 and, moreover, 
several guidelines recommend that nodules less than 5 mm 
without clinical risk factors for thyroid cancer should not 
undergo FNA even if suspicious malignant US features are 
observed. 31516 Among the 348 patients with 359 nodules, 79 
nodules in 76 patients were excluded because they showed 
cytologic results of nondiagnostic (n=50), atypia (n=13), 
suspicious for malignancy (n=6), and malignancy (n=10) 
without further cytopathologic diagnosis. Among the 150 
nodules with benign cytologic results, 71 nodules in 69 pa- 
tients were excluded because they did not undergo follow- 
up FNA or follow-up US and another 18 nodules in 18 pa- 
tients were excluded because they showed an increase in 
size in follow-up USs without further cytologic or patholog- 
ic evaluation. Finally, 191 thyroid nodules in 185 patients 
(mean age, 48.4 years; range 20-77 years) were included in 
this study (Fig. 1, Table 1). The patients included 153 wom- 
en (mean age, 49.5 years; range, 20-74 years) and 32 men 
(mean age, 50.7 years; range, 18-79 years). The mean nod- 
ule size (±standard deviation) was 11.1±7.4 mm (range, 5- 
51 mm). Among the 191 nodules, 139 nodules were con- 
firmed by operation (Surgery group) and the other 52 
nodules were observed by follow-up FNA or follow-up US 
after a year (Observation group). 



Observation group (n=52) 






Benign 


(n=52) 



Benign cytology, twice (n=8) 



Benign cytology and no interval 
change on F/U US for a year (n=34) 



Fig. 1. Diagram of study group. US, ultrasound; F/U, follow-up. 



191 nodules 



Surgery group (n=139) 



Benign (n=8) 



Benign cytology and decrease in 
size on F/U US for a year (n=1 0) 



Malignancy (n=131) 
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Table 1. FNA, BRAF , and Final Pathologic Diagnosis with Operation in Thyroid Nodules 





Number of nodules 


BRAF V600E mutation 


Pathology 


Nondiagnostic (n=6) 


1 
5 


Positive 
Negative 


PTC (n=l) 

PTC (n=4), completely ossified tumor (n=l) 


Benign (n=6) 


1 
5 


Positive 
Negative 


PTC (n=l) 

PTC(n=2),AH(n=3) 


AUS/FLUS (n=6) 


4 

2 


Positive 

Negative 


PTC (n=4) 
PTC (n=2) 


Suspicious for FN or FIN 
(n=6) 


0 
f. 

u 


Positive 

I'ICgaLlVC 


PTC fn=?1 FA (r\=V\ KW (vs=W 


Suspicious for malignancy 
(n=28) 


18 
10 


Positive 
Negative 


PTC (n=16), PTC, follicular variant (n=l), PTC, 
diffuse sclerosing variant (n=l) 

PTC (n=9), PTC, macrofollicular variant (n=l) 


Malignancy (n=87) 


74 
13 


Positive 
Negative 


PTC (n=72), PTC, follicular variant (n=l), PTC, 
diffuse sclerosing variant (n=l) 

PTC (n=13) 



PTC, papillary thyroid carcinoma; AH, adenomatoid hyperplasia; AUS/FLUS, atypia undetermined significance/fol licular lesion of undetermined signifi- 
cance; FN, follicular neoplasm; HN, Hurthle cell neoplasm; FA, follicular adenoma; FNA, fine-needle aspiration. 



Imaging analysis 

US imaging was scanned by radiologists with 1 to 13 years 
of experience. All US images were performed with a 5-12- 
MHz linear array transducer (iU22; Philips Medical Sys- 
tems). An investigator (J.Y.S.) selected two gray scale imag- 
es (both transverse and longitudinal scans) from each US 
examination on the Picture Archiving and Communication 
System (PACS, GE Medical System, Milwaukee, WI, 
USA). This investigator modified the images to jpg files 
and organized them on PowerPoint XP (Microsoft, Red- 
mond, WA, USA) slides in random order. She was exclud- 
ed from further image review after arranging the images. A 
total of 191 slides were made. Three radiologists (E.K.K., 
J.H.R, and J.H.S.) with 13, 17, and 9 years of experience in 
thyroid imaging, respectively, participated in this study. 
They were blinded to the cytologic analysis results and fi- 
nal pathologic diagnosis, and retrospectively reviewed the 
slides individually on a liquid crystal display monitor. 

All readers were asked to classify each nodule into two 
categories (positive for malignancy or negative for malig- 
nancy) using US features to decide whether the thyroid 
nodule should be aspirated or not. Readers were asked to 
designate a categorization for each nodule based on their 
overall impression of the nodule on US. In this study, read- 
ers were not asked to define US features such as internal 
content, nodule echogenicity, presence or absence of calci- 
fication, nodule shape, and nodule margin. 

US-guided Fine-Needle Aspiration (US-FNA) 

US-FNA was performed by the same radiologists who per- 



formed real-time US. Free hand US-FNA was performed 
with a 23 -gauge needle attached to a 2-mL disposable plas- 
tic syringe. Each lesion was aspirated at least twice. Ob- 
tained samples were expelled on glass slides, smeared, and 
placed immediately in 95% alcohol for Papanicolaou stain- 
ing. The remaining material in the syringe was rinsed in sa- 
line for cell block processing. Cytopathologists were not 
present during biopsies. 311 

Cytology slides were reviewed by an experienced patholo- 
gist to confirm the cytologic diagnosis. Based on the Bethes- 
da System for Reporting Thyroid cytology, FNA cytology 
results were categorized as nondiagnostic, benign, atypia 
undetermined significance/follicular lesion of undetemiined 
significance (AUS/FLUS), suspicious for follicular neo- 
plasm or suspicious for a Hurthle cell neoplasm, suspicious 
for malignancy and malignancy. 2 

DNA extraction and real time PCR 

The BRAF V600E mutation analysis was performed with 
DNA extracted from FNA cells remaining after cytologic 
evaluation. 

Real-time PCR was performed using the Applied Biosys- 
tems 7500 real-time PCR system (Applied Biosystems, Fos- 
ter City, CA, USA). The Real-Q BRAF V600E Detection Kit 
(BioSewoom, Korea) was used to cany out PCR reactions. 
The Real-Q BRAF V600E Detection Kit is a ready-to-use kit 
that detects the BRAF V600E (1799T>A) somatic mutation in 
the BRAF oncogene in a background of wild type genomic 
DNA with a multiplex real time PCR assay based on the 
TaqMan MGB probe system. The analytical sensitivity was 
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evaluated using the plasmid clone BRAF V600E mutation and 
the 95% positive cut-off value (limit of detection) was cal- 
culated at 21.5 copy/uL by the Probit analysis. 8 

Data and statistical analysis 

Final diagnoses of all malignant nodules were based on 
pathological results (n=131). Final diagnoses (n=60) of be- 
nign nodules were based on pathological results (n=8), with 
benign cytologic results being confirmed at least twice 
(n=8), a benign cytologic result and no increase in size 
(n=34), or a decrease in size being observed on follow-up 
US (n=10) (Fig. 1). 

Interobserver variability (10 V) of US was estimated us- 
ing the generalized kappa statistic of Landis and Koch. 17 
The degree of agreement was categorized in terms of kappa 
values: 0 corresponds to no agreement, 1.00 to complete 
agreement, less than or equal to 0.20 to slight agreement, 
0.21-0.40 to fair agreement, 0.41-0.60 to moderate agree- 
ment, 0.61-0.80 to substantial agreement, and 0.81-1.00 to 
near perfect agreement. 17 

To determine diagnostic performances, we used logistic 
regression with the generalized estimating equation or cate- 
gorical data analysis for repeated measures with the weight- 
ed least square method as appropriate. The US, BRAF V600E 
mutation, and cytology assessments were dichotomized ac- 
cording to the presence of malignant findings. The malig- 
nant cytologic diagnoses were considered positive cytology 
when calculating diagnostic values of FNA. When more 
than two diagnostic tests were combined, a nodule was con- 
sidered positive for malignancy when any test was positive. 
We calculated the diagnostic performances of cytology 
alone, cytology with US, cytology with BRAF V600E , and cy- 
tology with BRAF V600E and US. Also, we compared the di- 
agnostic performances of cytology, BRAF V600E , US, and 
combinations thereof. 

All statistical analyses were performed using SAS soft- 
ware (version 9.1.3; SAS Institute, Cary, NC, USA). Statis- 
tical significance was assumed for p-vahies less than 0.05. 
All reported /"-values are 2-sided. 



RESULTS 



Among the 185 patients, 126 patients (107 women, 19 men; 
mean age, 48.3; range 20-77 years) were included in the ma- 
lignant group and 59 patients (46 women, 13 men; mean age, 
48.4, range 28-72 years) were included in the benign group. 



The mean age and gender of the patients were not associat- 
ed with malignancy (p=0.569 and /?=0.244, respectively). 
There were 131 cases of malignancy (131/191, 68.6%) and 
60 cases of benign nodules (60/191, 31.4%). Benign nod- 
ules were significantly larger than the malignant nodules 
(mean size, 13.9±8.9 mm vs. 9.8±6.2 mm, respectively; 
/t=0.002). 

Reproducibility of US assessment 

The IOV of US assessment was calculated between the 
three readers. Reader 1 predicted 156 nodules as positive 
for malignant, reader 2 predicted 145 nodules as positive 
for malignant, and reader 3 predicted 101 nodules as posi- 
tive for malignant nodule. The generalized kappa value was 
0.242, indicating fair agreement was observed. Diagnostic 
performances varied between the three readers (Table 2). 

Comparison of diagnostic performance between the 
combination of cytology with BRAF V600E and cytology 
alone 

Of the 52 patients with negative cytologic results, 24 pa- 
tients harbored the BRAF V600E mutation. One of 6 patients 
with nondiagnostic cytology, 1 of 6 patients with benign 
cytology, 4 of 6 patients with AUS/FLUS cytology and 18 
of 28 patients with suspicious for malignancy cytology 
showed positive BRAF V600E mutation results (Table 1). 

The sensitivity, accuracy and negative predictive value 
(NPV) of cytology were 66%, 77%, and 57.7%, respec- 
tively (Table 2). When BRAF V600E and cytology were con- 
sidered in combination, the combination showed higher 
sensitivity (84.7%; /K0.001), accuracy (89.5%; jXO.001), 
and NPV (75%; /K0.001). Nevertheless, the positive pre- 
dictive values (PPVs) were the same between cytology 
alone and cytology with BRAF V600E . 

Comparison of diagnostic performance between the 
combination of cytology with BRAF V600E and the 
combination of cytology with US 

The combination of cytology with BRAF V600E showed in- 
creased specificity and PPV, compared with those of cytol- 
ogy with US (pO.OOl). In terms of accuracy, cytology with 
US showed decreased accuracy for all readers, but only 
reader 1 showed statistical significance, compared with that 
of cytology with BRAF V600E . However, the sensitivity of 
cytology with BRAF V600E was lower than that of cytology 
with US for all three readers (84.7% vs. 92.4%, 97.7%, and 
90.1%, respectively; p<0.005). 
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Comparison of diagnostic performance between the 
combination of cytology, BRAF V600E and US and the 
combination of cytology and US 

In terms of accuracy, PPV and NPV, the combination of cy- 
tology, BRAF V600E , and US showed increased diagnostic 
performances in all three readers compared to the combina- 
tion of cytology with US; however, only reader 1 and reader 
3 showed statistical significance. The combination of cytol- 
ogy, BRAF V600E , and US showed higher sensitivity (96.2%, 
98.5%, and 95.4%, respectively according to the reader) 
than the combination of cytology with US (92.4%, 97.7%, 
and 90.1%, respectively according to the reader) (/?<0.005). 

Comparison of diagnostic performance between the 
combination of cytology and BRAF V600E and the 
combination of cytology, BRAF V600E , and US 
The combination of cytology with BRAF V600E showed high- 
er specificity (100%) and PPV (100%) compared to the com- 
bination of cytology, BRAF V600E , and US (specificity 28.3%, 
66.7%, and 68.3%; PPV 74.6%, 86.6%, and 86.8%, respec- 
tively; /t<0.001). In terms of accuracy, the combination of cy- 
tology with BRAF V600E showed higher accuracy in all read- 
ers; only reader 1 showed statistical significance. However, 
cytology with BRAF V600E showed lower sensitivity (84.7%) 
than cytology with BRAF V600E and US (96.2%, 98.5%, and 
95.4%, respectively; /K0.001). 



DISCUSSION 



US-guided FNA is currently the standard diagnostic meth- 
od for patients with thyroid nodules. 1 - 3 However, up to 40% 
of cytologic results on thyroid nodules are inconclusive, 
which results in repeated aspirations and unnecessary surgi- 
cal interventions. 618 Such a high incidence of unnecessary 
procedures results in additional morbidity and higher medi- 
cal costs. 19 To resolve this issue, the scientific community 
has struggled to translate molecular markers into useful clin- 
ical tools for patients with thyroid nodules, and nowadays, 
several molecular markers are used to improve diagnostic 
accuracy in cases with inconclusive cytological results. 4,5 
Mutations or aberrant expressions of genes coding for sig- 
naling cascade proteins (RET, RAS, BRAF, PI3K, PTEN, 
AKT) have been identified in the majority of papillary thy- 
roid carcinoma (PTC) patients. 18 " 20 These alterations change 
the MAPK/ERK pathway and PI3K/Akt pathway, which 
play important roles in the transmission of cell signals and 



contribute to the transfomiation of malignant follicular cells. 
Several studies have demonstrated that analyzing the com- 
bination of BRAF, RAS, RET/PTC, and PAX8/PPARr muta- 
tions improves the diagnostic accuracy of thyroid cancer, 
particularly in samples with indeterminate cytology. 19 The 
BRAF V600E mutation is the most common mutation observed, 
and the frequency of the BRAF V600E mutation in PTC ranges 
from 29 to 83%. In Korea, which is a BRAF V600E mutation- 
prevalent area, the BRAF V600E mutation is present in more 
than 90% of PTCs. 21 The RAS mutation is the second most 
common finding, but in an FNA sample, it also can be de- 
tected in follicular carcinoma and other benign nodules. 
RET/PTC is found in 20% of adult sporadic PTCs. It usual- 
ly occurs in patients with a history of radiation exposure (50- 
80%) and in PTC from children to young adults (40-70%). 
In many clinical studies, its analysis is difficult and may only 
be useful in combination with other markers. These findings 
have made the BRAF V600E mutation a more generally ac- 
cepted reliable prognostic marker for PTC and as a result, 
additional molecular studies to improve the cytopathologic 
diagnosis of PTC have been focused on the BRAF V600E mu- 
tation. 4 * 22 

Various studies have supported the effectiveness of US 
for diagnosing malignancy in thyroid nodules. 1014 - 23 US as- 
sessment can be usefully adjusted in thyroid nodules be- 
cause it can reduce the false negative rate of FNA without 
additional medical cost. However, US is a rather subjective 
and operator dependent diagnostic method, and there have 
been some studies reporting various IO Vs in US assess- 
ments of thyroid nodules. 12 13 24 Therefore, it is hard to dis- 
cuss the diagnostic value of US as a additional method to 
FNA without a discussion about IOV In a previous study, 
moderate to substantial agreement was obtained in regards 
to IOV among a highly experienced group assessing thy- 
roid nodules, and the high interobserver agreement was 
thought to be obtained because the readers worked at the 
same institution, so they had been taught a uniform approach 
to translating sonographic findings of thyroid nodules. 12 In 
another study, the IOV among faculty members was higher 
than that of residents; the residents showed poor agreement 
for interpretation of US findings of thyroid nodules. 24 How- 
ever, in our study, only fair agreement was observed among 
highly experienced radiologists for interpretation of US 
findings of thyroid nodules. 

A recent study reported that the diagnostic performance 
of the combination of cytology, BRAF V600E and US was 
found to be superior to that of BRAF V600E with cytology in 
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terms of sensitivity, although, in terms of specificity and ac- 
curacy, results showed decreased diagnostic performance. 23 
However, the study did not include an analysis about the 
IOVs of US assessment. In this study, we investigated 
whether the combination of cytology, BRAF V600E , and US 
can be the best diagnostic method for detecting thyroid ma- 
lignancy, and if not, which combination is the most effective 
and objective ancillary method to FNA. To analyze 10 V, we 
evaluated US assessment conducted by three highly experi- 
enced radiologists who work at different institutions. We 
demonstrated a fair agreement of US assessment between 
the three highly experienced readers. This probably means 
that even experienced and specialized radiologists have 
their own diagnostic criteria based on their experience, thus 
this can cause low interobserver agreement between US as- 
sessments of thyroid nodules. Our results showed that the 
combination of cytology with BRAF V600E showed higher 
specificity, PPV, and accuracy, compared to the combina- 
tion of cytology with BRAF V600E and US. The combination 
of cytology with BRAF V600E and US showed increased sen- 
sitivity and NPV, compared with cytology with BRAF V600E , 
a conclusion similar to one from a previous study. 23 To be a 
good additional test, the tool should be highly reproducible 
as well as highly accurate. Considering the low reproducibili- 
ty of US and diagnostic performances, a combination of cy- 
tology with BRAF V600E can be a better diagnostic approach 
than a combination of cytology with BRAF V600E and US. 

There were some limitations to our study. First, there was 
a selection bias because we excluded thyroid nodules with- 
out further cytopathologic diagnosis or follow-up US to 
evaluate diagnostic performance. Second, we retrospective- 
ly reviewed still images specifically selected by an investi- 
gator, not real-time images. Therefore, these results may be 
different if the readers performed the US. Third, not all thy- 
roid nodules underwent surgery or cytopathologic diagno- 
sis. Some final references were based on cytology and fol- 
low-up US, so this can cause some false negative and false 
positive cytologic results. 3 - 25 Fourth, diagnostic perfor- 
mance varies according to the methodology of BRAF V600E 
testing; therefore, the result may not be reproducible if an- 
other method is used. Finally, the study population was 
made up of Korean patients, who are known to have a high 
prevalence of BRAF mutation; therefore, the conclusion of 
this study may not be relevant in other countries, especially 
in areas with a low prevalence of BRAF mutation. 

In conclusion, although the combination of cytology, 
BRAF V600E , and US can increase sensitivity, in terms of accu- 



racy and specificity, they can decrease the diagnostic perfor- 
mance. When also considering the low reproducibility of US, 
we concluded that the combination of FNA with BRAF V600E 
is the most reliable and objective method for diagnosing 
papillary thyroid cancer. 
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